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ABSTRACT 



A multi-node server transmits world-wide-web pages to 
network-based browser clients. A load balancer receives all 
requests from clients because they use a virtual address for 
the entire site. The load balancer makes a connection with 
the client and waits for the URL from the client. The URL 
specifies the requested resource. The load balancer waits to 
per form load balancing until after the location ofU he 
requested resour ce is known. The connection and URL 
request are passedlxom th e load balancer to a se cond node 
having the requested resource. The load balancer re-plays 
the initial connection packet sequence to the second node, 
but modifies the address to that for the second node. The 
network software is modified to generate the physical net- 
work address of the second node, but then changes the 
destination address back to the virtual address. The second 
node transmits the requested resource directly to the client, 
with the virtual address as its source. Since all requests are 
first received by the load balancer which determines the 
physical location of the requested resource, nodes may 
contain different resources. The entire contents of the web 
site is not mirrored onto all nodes. Network bottlenecks are 
avoided since the nodes transmit the large files back to the 
client directly, bypassing the load balancer. Client browsers 
can cache the virtual address, even though different nodes 
with different physical addresses service requests. 

16 Claims, 18 Drawing Sheets 
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WORLD-WIDE-WEB SERVER WITH Internet-Protocol (IP) address. Each computer is typically 

DELAYED RESOURCE-BINDING FOR assigned a different IP address so that no two machines have 

RESOURCE-BASED LOAD BALANCING ON the same IP address. The IP address is often written as four 

A DISTRIBUTED RESOURCE MULTI-NODE decimal numbers separated by periods. Each decimal num- 

NETWORK 5 ber represents an 8-bit binary number, from zero to 255 in 

decimal notation. Thus a computer in IBM's domain might 

BACKGROUND OF THE INVENTION — FIELD have the IP address 209.180.55.2 while another computer in 

OF THE INVENTION that domain might have the address 209.180.55.103. 

m . . . . , , . j Client Browsers Accessing Web Servers 

This invention relates to network servers, and more par- 1 . „ c %• „ i i i„* *v m 

*cul 1 to Internet Se rs io FIG. 1 is a diagram of a client browser looking up the IP 

^ address of a host specified in a URL. Users of a remote 

BACKGROUND OF THE INVENTION computer use client software known as an Internet browser 

DESCRIPTION OF THE RELATED ART or s^ply a browser. Popular browsers include Netscape 

Navigator by Netscape Communications, Inc. of Mountain 

Use of the global network known as the Internet has 15 Mew, Calif, and Internet Explorer by Microsoft Corp. of 

skyrocketed. Advertisers commonly feature their Internet Redmond, Wash., although many other browsers and other 

addresses in television, billboard, and magazine ads. Con- types of client software are used. 

sumers with a remote computer can access the Internet using Browser 10 initiates a communication session with a 

client software known as a browser. Explosive growth is remote server by the user selecting a URL, perhaps by 

occurring in the part of the Internet known as the World- 20 mouse-clicking on a hyper link to a new web page. Host 

Wide Web, or simply the "web". The web is a collection of name 11, "www.^ound.com , ^ in the URL "http:// 

millions of files or "web pages" of text, graphics, and other www.round.com/file.html M , is sent to domain-name-system 

media which are connected by hyper-links to other web (DNS) server 14, which is a special Internet server with 

pages. These may physically reside on a computer system look-up table 16. DNS server 14 is often a special server at 

anywhere on the Internet — on a computer in the next room 25 an Internet Service Provider which contains most or all 

or on the other side of the world. domain names on the entire Internet, or in a local region of 

These hyper-links often appear in the browser as a graphi- the Internet One DNS server may have to refer the request 

cal icon or as colored, underlined text. A hyper-link contains to another DNS server for unknown host-names, 

a link to another web page. Using a mouse to click on the DNS server 14 looks through look-up table 16 and finds 

hyper-link initiates a process which locates and retrieves the 30 an entry for the host www.round.com. This entry contains a 

linked web page, regardless of the physical location of that physical IP address 18 for the web-server host in the domain 

page. Hovering a mouse over a hyper-link or clicking on the round.com. This IP address 18 230.101.17.101 is returned to 

link often displays in a corner of the browser a locator for the browser 10. Browser 10 then stores this IP address in client 

linked web page. This locator is known as a Universal cache 20 for future use, a process known as browser caching 

Resource Locator, or URL. 35 of the IP address. 

Background of URL's, IP Addresses, HTML, HTTP Browser 10 then uses cached IP address 18' to initiate a 

The URL identifies a domain, a host within that domain, communication session with the remote computer which 

and sometimes a resource or file within a directory structure physically has the desired web page, the www.round.com 

on the host computer. Domains can be thought of as a group server having the file .html file. FIG. 2 shows a browser using 

of computers, such as all computers on a company's net- 40 a cached IP address to retrieve a file from a remote server in 

work. For example, the domain "ibm.com" identifies a a server farm. Browser 10 reads the cached IP address 18* 

domain for the commercial company IBM, which may from client cache 20 and uses cached IP address 18* to 

include thousands of individual computers. Typically the initiate a communication session with remote server 22. 

URL identifies only those computers which are servers on Once the session with server 22 is established, URL 12 is 

the world-wide web by prefixing the domain with a host 45 sent to server 22. Server 22 then accesses disk 24 which 

name. Thus the URL "http://www.ibm.com" identifies an includes requested file 26, the file.html web page. A file copy 

individual host computer within the ibm.com domain which 26* of requested file 26 is sent back to browser 10, which 

operates as a world-wide-web server for IBM. "HTTP" tells re-constructs the web page from file copy 26* and displays 

the host to use the hyper-text transfer protocol while deliv- the web page to the user. Other files such as graphic image 

ering files over the Internet. The files delivered can be from 50 files may also be transferred which were not directly 

resources such as database queries or execution of scripts by requested by the URL, but are referenced by the file.html 

the host as well as traditional files. file. 

A web server site may contain thousands of individual Server Farms for Large Web Sites Mirror Content 

web pages. The location of the file or resource containing a While some smaller web sites can be served from a single 

desired page is identified by appending a directory-path file 55 computer, larger web sites require multiple computer 

name to the host and domain names in the basic URL to form machines acting as servers. Some web sites receive as many 

a new URL. Thus the URL "http://www.ibm. com/dira/dirb/ as one million requests or "hits" per hour, requiring many 

dirc/intro.html" identifies a hyper-text markup-language workstation computers. 

(HTML) file called "intro.html" which resides on a host FIG. 2 shows server farm 30 which contains server 22 

named "www" within the ibm.com domain. The file resides 60 serving browser 10, and servers 22A, 22B, 22C which are 

in the dira directory and the dirb/dirc subdirectory. Often this servicing other browsers (not shown). Servers 22 A, 22B, 

HTML file contains references to other files which are 22C each contain their own disks 24', each with a copy of all 

loaded automatically by the client's browser. the web pages in the site, including requested file 26. Server 

While the URL is used to locate a file on a host within a farm 30 is basically a group of replicated servers which can 

domain, it does not contain a physical address for the host 65 service requests from multiple browsers. Each server has a 

computer. Addresses of computer machines on the Internet copy of the entire web site. Any server can service any 

are specified using a 32-bit numeric identifier known as the request since the content is "mirrored" on all servers. 
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Each machine typically has its own unique IP address. 
Since a domain can have many computer machines with 
many IP addresses, some way to provide to a client one of 
the many server machines' IP address is needed. One simple 
approach is known as rotating DNS or DNS round-robin 
load-balancing. 

DNS server 14 of FIG. 1 contains look-up table 16 which 
is used to return IP addresses to host-lookup requests from 
client browsers. Look-up table 16 contains entries for dif- 
ferent host names. The entry for a host name specifies the IP 
addresses for that host and each entry can contain several IP 
addresses for that host. The entry for www.round.com host 
on the domain round.com contains four IP addresses: 

230.101.17.100 

230.101.17.101 

230.101.17.102 

230.101.17.103 
for the four servers 22A, 22, 22B, 22C of server farm 30 
serving the www.round.com web site. When a client 
requests a DNS look-up, one of these IP addresses is chosen 
in a round-robin fashion. Each time a different client looks 
up the host www.round.com, a different IP address is 
returned until all the available IP addresses are used. Then 
the first IP address is returned again. Thus the first browser 
is sent the IP address for server 22A, the second browser is 
sent the IP address for server 22, the third browser sent the 
IP address for server 22B, and the four browser sent the IP 
address for server 22C. The fifth browser request to DNS 
server 14 is sent the first server 22 A, and so on in a 
round-robin fashion. 

Each DNS server operates independently of other DNS 
servers. Thus optimal load balancing is not always achieved. 

Other more sophisticated assignment schemes have been 
used, such as "load-balancing DNS" which sends requests to 
servers based on a balancing algorithm which attempts to 
balance the load on each server. With this approach more 
powerful servers could be assigned more requests than 
weaker servers. 
IP Addresses of Servers Cached on DNS Server 

DNS servers 14 (FIG. 1) often cache the results of 40 
domain-name lookups which were passed or forwarded to 
other DNS servers for completion. The adrninistrator of the 
www.round.com web site has no way of actively updating 
the contents of many DNS caches containing IP addresses of 
servers in server farm 30. Instead, the administrator must 
rely on the remote DNS servers periodically flushing their 
own cached IP addresses and looking up the www.round- 
.com host again. DNS servers may flush their cached IP 
addresses every few minutes or not for several weeks. IP 
addresses can thus remain in a DNS server's cache long after 
the server with the cached IP address is removed from 
service. The IP address of the removed server can continue 
to be assigned by the DNS server until the cached entry is 
replaced or flushed. 

For the example in FIG. 3, when server 22C crashes, its 
IP address 230.101.17.103 remains in use in DNS server 
caches. Users that look-up the www.round.com host name 
can be assigned the IP address of crashed server 22C. Users 
sent the IP address of crashed server 22C are unable to 
access server farm 30, even though several other servers 
22A, 22, 22B at server farm 30 are operational. 
DNS Caching Blocks Some Users From Partially-Crashed 
Web Site 

Several hours or even days may be required to flush the 
IP address of the crashed server 22C from all DNS caches. 
Thus DNS servers can continue to send the IP address of the 
crashed server to browsers long after the server has crashed. 



30 
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Browsers attempting to use this IP address and connect with 
the crashed server receive no response from the www.round- 
.com web site. These browsers are frozen out of the 
www.round.com web site. 

Since the browser itself caches the IP address from the 
DNS server until the browser application is closed, browsers 
can still attempt to access a crashed server after the crash has 
occurred. FIG. 3 shows a browser using a cached IP address 
to access a crashed server which is not responding. Browser 
10A had previously cached IP address 18C for server 22C 
for the www.round.com host. When browser 10A attempts to 
connect to wwwjound.com, server 22C is accessed. No 
response is received from server 22C since the server is not 
functioning. To Browser 10A, the web site www.round.com 
appears to be non-functional, even though to another 
browser 10, the web site is functional. 

Though the user of browser 10A may repeatedly try to 
connect to the www.round.com web site, each time no 
response is received until server 22C is fixed. Since DNS 
server 14 of FIG. 1 may continue to use the IP address of the 
crashed server 22C, many users may be locked out from the 
web site, even though other users can access the site. 

When browser 10A also caches IP address 18C, the 
browser may not be informed that the IP address is no longer 
valid even after DNS server updates its own cache. These 
browser caches may persist for several hours, preventing the 
user from accessing the web site. Should the server 22C be 
removed from service permanently, perhaps being 
re-assigned to another web site, the user is effectively 
blocked from accessing the web site until the user flushes his 
IP cache, which may not occur until the user exits the 
browser application. 

Of course, with a large server farm, the loss of one server 
blocks out only 1/N of the users, where N is the number of 
servers in the server farm. Thus for FIG. 3, one-fourth of the 
current users are blocked out while %ths of the current users 
have access to the web site. One-fourth of the new users 
looking up the host on a DNS server which still uses the old 
IP address of the crashed server are also blocked from the 
web site. 

Router-Based Web Site 

An approach which mitigates some of these problems 
inserts a multiplexer or router between the browser clients 
and the server farm. FIG. 4 illustrates a router-based server 
farm. A single IP address of router 32, 230.101.17.200, is 
available to all DNS servers as the single IP address for the 
web site. Browser 10 caches this IP address as cached IP 
address 34. Requests from browser 10 are sent to router 32 
since cached IP address 34 points to router 32. 

Router 32 receives all packets in the transmission from 
browser 10. Router 32 might be a dedicated personal com- 
puter (PC) which uses an algorithm to determine which of 
servers 36A, 36, 36B, 36C in server farm 38 should service 
the request from browser 10. Router 32 may use a fairly 
complex load-balancing scheme which takes into account 
requests from other browsers and the capability of each 
server when some servers are powerful workstations while 
other servers are older, slower PC's. 

All the packets in the session from browser 10 received by 
router 32 are re-transmitted to server 36, with the destination 
IP address changed to the IP address for server 36, 
230.101.17.101. Server 36 retrieves the requested file 26 
from its local disk 24 and transmits it back to router 32, 
which then re-transmits the file to browser 10. 

When a server crashes, such as crashed server 36C, only 
those browsers which are currently connected to server 36C 
experience server failure. Client caching of the router's IP 
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address causes all new sessions to be routed to router 32; that fewer users experience a browser lock-up when a server 

only sessions in progress to crashed server 36C receive no at the web site fails. A web site that can use the standard 

response from the web site. Thus when one of the servers DNS mechanism is desired to overcome the limitations of 

fails, only 1/N of the currently active requests fail, where N DNS caching and complex maintenance of round-robin 

is the number of servers. New requests do not fail since 5 DNS. 

router 32 detects when crashed server 36C is not functioning . „ 

and no longer directs new requests to the down server. SUMMARY OF THE INVENTION 

A commercial embodiment of a router-based web server A web site sends resources to a browser on a client 
has been announced by SOS Corp. of New York, N.Y., under connected to a computer network. The web site has a 
the name "HydraWEB", and product literature indicates that id network connection point for receiving incoming data pack- 
a patent is pending. A second commercial embodiment is the ets from the computer network and for transmitting outgoing 
Cisco Local Director, manufactured by Cisco Systems of data packets to the computer network. A local network is 
San Jose, Calif. Each server 36A, 36, 36B, 36C contains a coupled to the network connection point and transfers data 
local copy of all content on the web site on disks 24, 24*. packets. A plurality of network nodes contain web servers 
Mirroring the full content of the site to all servers is a is with resources. The plurality of network nodes is connected 
disadvantage for web sites with a large amount of content, to the local network. Hie plurality of network nodes transmit 
because of the size and cost of the local disks. Certain web the resources as outgoing data packets over the local net- 
applications such as multimedia and video delivery can work to the network connection point through the computer 
require a particularly large amount of disk space. These network to the client. 

applications are expensive to implement and thus minimiz- 20 a balancer network node contains a load balancer that 

ing the number of copies at the server farm is desirable. receives the incoming data packets transmitted over the local 

Another disadvantage with the router web site is that all network from the network connection point. The load bal- 

data transfers go through router 32. Since many web pages ancer determines an assigned server in the plurality of 

contain graphics or even video or sound, the amount of data network nodes to respond to a request from the client 

transferred from the server through the router to the browser 25 contained in an incoming data packet. The load balancer 

is large. Router 32 must be fast and efficient to handle load transfers a connection to the client to the assigned server, 

balancing androuting of incoming and outgoing packets. As ^ balafloer aetW0fk node wntainin me load balancer 

the web site becomes more popular and traffic grows, router fa 00mected to mc nctwork connection point by the local 

32 can quickly become a bottleneck and limit performance netWQrk which ^ ^ coanected to the luralit of network 

of the web site. Router 32 is also a single ^point ^of failure. 30 nodes ^ me ^ ^ are r011ted to the 

Load-Balancing Granularity Determines Users Affected by balance r network node but outgoing data packets bypass the 

Server Failure balancer network node. 

For round-robin DNS, the IP address of the web server is T . t ..... _ . - . . * 

, . 4 . V . u j it « . In further aspects the balancer network node is in the 

assigned once to the client browser and all subsequent , c i . • * . ™ . 

accuses use this IP address until the browser's clientele 35 P*"? 1 * * aetw ^ k ?° des contomng web servers. The web 

» t*i_ * * i- * i i i j l i • i * site is addressable by one network address for all web 

is purged. This is client-level load-balancing granularity, . . t r ^ * j . . . ,_ 

, iT^ r ii servers m the plurality of network nodes containing web 

since each client is assigned one server machine for all . . J , . At _ . r ^ . e . 

requests from that client. When the assigned server crashes, Ea f ^tworknode m the plurality of ne^ork nodes 

the clients using that server are blocked for all future c ? rtains ^ * £ ortK * [ ot * ™° uxces ^^ti J 

requests until thl client application is closed. 40 ^TJf at atC m n0t nnno,cd l ° aU QCtW ° rk 

The router-based web site has request-level load- nodes al ine weD Slte ' 
balancing granularity. Servers are assigned to handle incti- In sti11 further aspects of the invention a content means 
vidual requests from browsers. When the assigned server storcs an indication of which network nodes in the plurality 
crashes, the outstanding requests to the server are blacked of network nodes contain each resource. A URL means 
out but clients can still access other servers in the farm. 45 receives incoming data packets from the client which con- 
Server Problems Plague the Internet tain a ™l uesi for a resource. A requested resource is deter- 

Many Internet users can testify to the utter frustration from ™ e incoming data packets. Compare means is 

when the "SERVER NOT RESPONDING" error message is coupled to the content means and is coupled to the URL 

displayed on their browser while trying to connect to a web means. It compares the requested resource to the indication 

site. Users often blame the company which administers the 50 of which network nodes in the plurality of network nodes 

unavailable web site. Web sites are not as fault-tolerant as each re souroe. A list of network nodes containing the 

possible despite large investments in replicated servers. An requested resource is outputted. 

intelligently-designed web-site architecture with better Balancing means receives the list of network nodes con- 
fault-tolerance is needed. taining the requested resource. It chooses as an assigned 
It is desired to reduce the frequency of "SERVER NOT 55 node one of the network nodes in the list of network nodes. 
RESPONDING" messages that Internet users often receive. Thus the load balancer chooses an assigned node based on 
While many web sites use server architectures such as DNS me resources contained by each network node. The load 
round-robin and router-based load-balancing, a more effi- balancer performs resource-based load balancing, 
cient and fault-tolerant web-site architecture is desired. It is In other aspects a delay means in the load balancer delays 
desired to avoid the data bottleneck and single point of 60 assignment of the assigned node until an incoming data 
failure at the router for router-based web sites. It is also packet containing the request for the resource is received, 
desired to use inherent characteristics of web traffic to more Thus load balancing is delayed. 

efficiently design a web-site architecture. Mirroring the In other aspects the invention is a method which makes a 

content of the entire web site to all servers at the site is connection and sets up a session between the client and a 

undesirable, but having differing content on different servers 65 load balancer at a web site. TTie load balancer waits for a 

is desired while still performing load balancing. A web site URL request from the client once the load balancer has made 

with request-level load-balancing granularity is desired so the connection with the client. It receives the URL request 
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from the client and decodes the URL request to determine a FIG. 13 is a diagram of network layers transmitting 

requested resource. An identifier for the requested resource packets from the server to the client browser. 

is compared to identifiers for resources located on a plurality FIG. 14 is a diagram of modifications to the IP layer for 

of nodes and a first subset of the plurality of nodes is the load balancer's node. 

determined which contain the requested resource. The URL 5 1C . „ a . M . c _ A . a j m i 

. j , • 1 j • i_ is * l * r FIG * 15 is a flowchart for a modified IP layer input 

request is assigned to an assigned node in the first subset of module 

the nodes which contain the requested resource and the „ IO ' . n . T¥> , 

connection and the session setup are transferred to the . ^1 ^ flowchart of an IP layer output module which 

assigned node which contains the requested resource. The 15 modlfied for ^ansmitUng packets from the load balancer, 

assigned node reads the requested resource and transmits the 10 mG - 17 highlights that the real IP address of the assigned 

requested resource to the client. Thus the assigned node is server * uscd whcc multiple hops are required, 

selected based on a location of the requested resource FIG. 18 is a flowchart of the operation of the load 

determined from the URL request. balancer. 

In further aspects the packets received from the client are ^ & a diagram of a fault-tolerant web site with a 

TCP/IP packets having a destination IP address which is a 15 back-up load balancer and dual Internet connections, 
virtual IP address of the load balancer. The virtual IP address 
of the load balancer is changed in the packets to a real IP 

address of the assigned node and the packets are passed to The present invention relates to an improvement in Inter- 

a modified IP layer. The real IP address determines a net and Intranet server farms. The following description is 

physical route from the load balancer to the assigned node 20 presented to enable one of ordinary skill in the art to make 

over a network and a physical network address is generated and use the invention as provided in the context of a 

for the assigned node and the physical network address is particular application and its requirements. Various modifi- 

attached to the packets. The real IP address in the packets is cations to the preferred embodiment will be apparent to 

changed back to the virtual IP address of the load balancer those with skill in the art, and the general principles defined 

before transmission of the packets with the physical network 25 herein may be applied to other embodiments. Therefore, the 

address. Thus the physical network address is generated present invention is not intended to be limited to the par- 

from the real IP address of the assigned node, but the packets ticular embodiments shown and described, but is to be 

are transmitted to the assigned node containing the virtual IP accorded the widest scope consistent with the principles and 

address of the load balancer. novel features herein disclosed. 

30 The inventors have realized that most traffic at web sites 

BRIEF DESCRIPTION OF THE DRAWINGS ^ inherently asymmetric. Users download huge amounts of 

FIG. 1 is a diagram of a client browser looking up the IP data md &*P*x* from a web site > b ^t provide only small 
address of a host specified in a URL. amounts of data in the form of requests to the web site. The 

™„ - , , . , - _ , , amount of data flowing out of a web site far surpasses the 

FIG. 2 shows a browser using a cached IP address to 35 ^ of ^ flowin ^ site . 

retrieve a file from a remote server m a server farm. r ^ „ 

~ , , . L j T „ FIG. 5 highlights the asymmetric nature of data transfer at 

FIG. 3 shows a browser using a cached IP address to a mldMjm3b site . Ghent browser 10 operates an Inter- 

access a crashed server which is not responding. m browsef application which connects t0 ^ ^ h 

FIG. 4 illustrates a router-based server farm. server farm 30. Client browser 10A also operates an Internet 

FIG. 5 highlights the asymmetric nature of data transfer at 40 browser application which connects to server 22C in server 

a world-wide-web site. farm 30, and other browsers (not shown) may also be 

FIG. 6 is a diagram of a web server which asymmetrically connected to this and other servers 22A, 22B. 

routes incoming traffic through a load-balancer while Browser 10 imbeds requests and commands and a small 

bypassing the load-balancer for files transmitted back to 45 amount of data in URL's, which are transmitted to server 22. 

client browsers. Each URL contains about 50 to 150 bytes of information, 

FIG. 7 is a diagram of a TCP/IP packet transmitted excluding IP addresses and packet headers and other net- 
through a local-area network (LAN) work overhead. A URL often contains information other 

FIG. 8 is a diagram illustrating TCP state migration of a * an a For example, when the user of browser 

connection from the load balancer to a server node. 50 10 *™**<te** °° ^ bitmap image disp ayed on a web 

n . _ . page, the relative coordinates or the mouse s location when 

FIG. 9 is a flowchart of processing a browser request by ^ mouse dick occurred are included in the URL: http:// 

a pnor-art router-based load balancer such as shown in FIG. www.round.com/cgi-bin/coo.cgi7102.315. Server 22 

decodes the coordinates in the URL and determines where 

FIG. 10 is a flowchart showing load balancing and state 55 on the web page the user mouse-clicked, 

migration delayed until the connection is made and the URL ^ then perform the action requested, such as 

request received opening and transmitting another web page to browser 10. 

FIG. llAis a chart illustrating packets transferred among other information sometimes embedded in the URL 

the browser, load balancer, and the assigned server when includes passwords or search text that the user types in, or 

establishing the connection and transferring the connection 60 name and address information typed in. Since the amount of 

to the assigned server which responds to the URL request. data that a user types in is limited, the number of bytes for 

FIG. 11B shows the browser sending a second URL this data is small, 

request, PUSH(l), to the load balancer, which is passed Traffic from server 22 to browser 10 consists of large data 

through to the assigned server as PUSH(l)'. files which are used to reconstruct the web pages on browser 

FIG. 12 is a diagram of network layers showing a packet 65 10. Since many web sites feature colorful graphics, large 

sent from the client to the server which is intercepted by the graphics files are typically transmitted from server 22 to 

load balancer. browser 10. As new types of media become commonplace, 
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sound, animation, and video files will accompany the graph- IP addresses altered since each server uses the virtual IP 

ics and HTML files being transmitted from server 22 to address as the source address of outgoing packets. Thus the 

browser 10. Smaller web pages may result in only 10 larger outgoing traffic of large mega-byte-sized files is 

K-bytes of data returned to browser 10, while more graphi- bypassed around load-balancer 54. Multiple connections to 

cal web pages cause larger files to be transmitted, perhaps to 5 the Internet can be provided for the outgoing traffic, since a 

several mega-bytes. Video and sound file can easily exceed single router/load-balancer is not required for all traffic, 

several mega-bytes. NIC Address Encapsulated in TCP/IP Packet 

Hie result is that only 50 to 150 bytes of data are typically FIG. 7 is a diagram of a standard TCP/IP packet trans- 
transmitted from browser 10 to server 22, while tens or mitted through a local-area network (LAN). Packet 180 
hundreds of thousands of bytes of data are transmitted in the 10 contains NIC address 182, which contains the low-level NIC 
reverse direction, from server 22 to browser 10. In the near address (e.g. Ethernet's media-access-controller (MAC) 
future, when sound and video become common, the reverse address) of the next destination station in the route to the 
traffic will increase to several mega-bytes per 100-byte final destination indicated by destination IP address field 
request. At that point the bandwidth of the reverse traffic will 186. Several destination stations may exist on the route to 
be about a thousand times the bandwidth from browser to is the station having the destination IP address. Each interme- 
server. diate station reads the destination IP address and determines 
Outgoing Data Bypasses Load-Balancer the next station in the route. The NIC address of this next 

FIG. 6 is a diagram of a web server which asymmetrically station is changed to the NIC address for the next leg of the 

routes incoming traffic through a load-balancer while route, but the IP addresses are not changed, 

bypassing the load-balancer for data transmitted back to 20 The source station's IP address is contained in source IP 

client browsers. Browsers 10, 10A cache a virtual IP address address field 184, while the packet's type or protocol is 

34 in client cache 20. Virtual address 34 (230.101.17.200) is included in protocol field 188. For Internet packets, the 

an IP address that identifies all servers at the web site. Unlike protocol is TCP, but other protocols may be used for local 

a conventional IP address which is unique to an individual LAN traffic. The data being transmitted by the packet is 

host or server, the virtual IP address identifies the web site 25 contained in data field 189. A frame checksum (not shown) 

in its entirety. may be appended. Source IP address field 184, destination IP 

External routers are configured to deliver all incoming address field 186, and protocol field 188 are the IP header 

URL's and network traffic from browsers 10, 10A to load- attached to the data by the IP layer, while NIC address 182 

balancer 54, which has the virtual IP address. is attached by the data-link layer. Packet 180 includes a 

Load-balancer 54 keeps track of which requests are being 30 TCP/IP header which includes flags indicating the type of 

processed by each server in server farm 50, and attempts to packet (SYN, ACK, PUSH, RST, FIN). A sequence number 

balance the load of requests among the servers. As is is also included in the TCP/IP header to keep track of 

subsequently discussed in more detail, load-balancer 54 packets received 

establishes the connection with browser 10 and waits for the Larger data files must be broken into several packets 

URL before performing load balancing and assigning the 35 which are transmitted separately and then re-assembled. A 

request to a server. The connection and the URL request is typical Ethernet packet can contain up to 1500 bytes, 

then migrated to the assigned server. For example, the TCP Connection Migration — FIG. 8 

request from browser 10 is assigned and migrated to server FIG. 8 is a diagram illustrating TCP state migration of a 

52, while the request form browser 10A is assigned and connection from the load balancer to a server node. Browser 

migrated to server 52C. 40 10 connects through Internet 66 to load balancer 70 and 

Unlike a router-based web site, the IP addresses of packets sends a URL request 102 once the connection 100 is made, 

are not changed to the assigned server's local IP address. Load balancer 70 does not have to be a separate, dedicated 

Instead, each server 52A, 52, 52B, 52C is assigned an router or PC, and is shown as a software application running 

additional IP address, the virtual IP address. A low-level on server 56. Load balancer 70 can use many variations of 

Network-Interface Card (NIC) address is used to route the 45 balancing algorithms to determine which server 56, 51, 52 

packets to the assigned server. Each server accepts any should service the new URL request 102. Load balancer 70 

packet with either its local IP address or the virtual IP determines that the request should be assigned to server 52. 

address. Incoming packets from the Internet backbone are The connection and URL request are migrated from load 

routed to load-balancer 54 because they are given the NIC balancer 70 to server 52 using TCP state migration 120. 

address of load balancer 54 by the Internet connection router so Server 52 accesses disk 62 to read requested file 26 and 

(not shown). sends a copy of requested file 26 to browser 10 through 

Server 52 sends HTML files for the web page back to Internet 66 as data transfer 104. 

browser 10 by using the source IP address in the packets Resource-Based Load Balancing 

received from load-balancer 54. Since load-balancer 54 does Each server 56, 51, 52 has a local disk 58, 60, 62, which 
not modify the higher-level TCP/IP packets transmitted to 55 contains different content. For example, only disk 62 con- 
server 52, server 52 appears to be receiving these packets tains requested file 26. Load balancer 70 maintains a direc- 
directly from browser 10. Server 52 uses the browser's IP tory table of the locations of different files and resources on 
address as the destination IP address for outgoing packets, the web site. Load balancer 70 determines that only server 
but the virtual IP address as the source address of these 52 and not servers 56, 51 can handle the request. The entire 
packets. Outgoing packets do not go through load-balancer 60 content of the web site does not have to be mirrored to each 
54. The bandwidth of traffic through load balancer 54 is server's hard disk as in the prior art. Some of the more 
much less than through router 32 of FIG. 4 since only the frequentiy accessed files and resources, such as the home 
relatively small incoming requests are routed through load page, may be replicated to all servers, but less-frequently 
balancer 54 while outgoing data bypasses load balancer 54. accessed files and pages may be located on a single node or 
Load balancer 54 can handle multiple requests from 65 a few nodes. Other web sites resources may include dedi- 
multiple browsers since each request is relatively small. cated servers with specific resources such as databases of 
Outgoing packets from each server do not have to have their built-in application-programming interfaces (API's) to inter- 
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face with user-defined programs, or software licenses to run received are stored by the load balancer and then played 

particular programs on particular servers. back to the assigned server. The assigned server accesses its 

Prior Art Load Balancing Performed Before URL is Parsed local disk to read the requested file and sends a copy of the 

FIG. 9 is a flowchart of processing a browser request by requested file to the browser through the Internet as data 

a prior-art router-based load balancer such as shown in FIG. 5 transfer 104. 

4. The browser first attempts to make a connection with a TCP State Migration — FIG. 11A 

server at the web site by sending a SYN packet which FIG. 11A is a chart illustrating packets transferred among 

requests that a connection be made, step 150. A SYN packet the browser, load balancer, and the assigned server when 

has its SYN flag set in the TCP header. The router/load establishing the connection and transferring the connection 

balancer receives this request and performs load balancing 10 to the assigned server which responds to the URL request. A 

based on the load of each server in the server farm. Load browser application running on a remote client initiates a 

balancing selects an assigned server for this new request, connection by sending a synchronizing packet, SYN(0), to 

step 152. The router/load balancer then forwards packets to the virtual IP address of the web site. Hie SYN(0) packet is 

the assigned server, step 154. routed to the load balancer since it is an incoming packet, 

The assigned server then makes the connection with the 15 and the load balancer sets up a session with the browser by 

browser, step 156, by returning an SYN/ACK acknowledge setting aside memory space and creating an entry in a 

packet to the browser by first routing it through the router. session table. TTie load balancer replies with a SYN/ACK 

The browser responds with an ACK packet and then with a packet to the browser, and the browser replies with an 

URL request, step 158. This URL packet is received at the acknowledgment packet, ACK(0). The SYN packet contains 

router and then re-transmitted to the assigned server. The 20 an initial sequence number which is determined by the 

assigned server parses the URL request to determine which browser's OS. The SYN/ACK packet contains acknowledg- 

files are requested and then transmits these files back to the ment number which is this initial sequence number incre- 

browser through the router/load balancer, step 160. mented by one. At this point connection 100 has been 

Steps 150, 152, 154 are performed by the router/load established between the browser and the load balancer, 

balancer while steps 156, 158, 160 are performed by the 25 The load balancer saves all of the SYN and ACK packet 

assigned server. The router passes packets from the server information received. Since incoming packets are few and 

through to the client's browser. small in size, a large amount of storage is not necessary. The 

URL File Request Arrives After Load Balancing in Prior Art SYN and ACK packets are later played back to the assigned 

Since load balancing is performed as soon as the first SYN server for TCP state migration, 

packet is received, before the URL is sent, such prior-art 30 The browser then sends the first URL request 102 in a 

load balancing cannot take into account the resource or file PUSH packet. A PUSH packet is identified by a PUSH flag 

which is requested by the URL. All servers must have the being set in the TCP header. The load balancer parses the 

same content, since the URL arrives after the server assign- URL to get the file or resource name. Based on the requested 

ment has been made. The invention overcomes these limi- resource, and the location of each resource in the web site, 

tations by delaying the assignment of the server until after 35 the load balancer determines which servers can serve the 

the URL has been received. request, and then chooses the least busy of these servers as 

Simple router-based load balancing does not allow for the assigned server, 

resource -based load balancing because the requested The load balancer then transfers the condition or state of 

resource or file is not known when the connection is first the connection to the assigned server in a process the 

made. The requested file is known once the URL is received 40 inventors call "TCP state migration". TCP state migration is 

and parsed, which occurs after the connection is made. The not visible to the browser since all transactions occur 

browser does not send the URL until the connection has between the load balancer and the assigned server. The 

been made and the server responds with an acknowledgment assigned server also uses the virtual IP address so the 

packet. The router approach of FIGS. 4, 9 forwards all browser is not aware that the SYN/ACK came from the load 

packets to the assigned server, and the assigned server then 45 balancer while the data comes from the assigned server, 

makes the connection and sends the acknowledgment. Once TCP state migration 120 is performed by the load balancer 

the connection is made with the assigned server, then the playing back the SYN packet received from the browser and 

browser sends the URL, which is forwarded by the router to stored by the load balancer. The server responds with a 

the assigned server. SYN/ACK packet, which is intercepted and directed to the 

Delayed Load Balancing and TCP State Migration 50 load balancer and not sent to the browser. The load balancer 

FIG. 10 is a flowchart showing load balancing and state then sends the browser's stored ACK packet to the assigned 

migration delayed until after the connection is made and the server, and the assigned server is then connected directly to 

URL request received. The TCP connection 100 is made the browser, having the same TCP state as was established 

between the browser and the load balancer by exchanging with the load balancer. 

SYN and ACK packets. Once this connection is made, the 55 The load balancer then sends the packet(s) containing the 

load balancer waits until the browser sends the URL which URL request to the assigned server as packet PUSH(0)'. The 

indicates which file or resource is requested. Once the URL server reads its local disk and sends the requested file to the 

request 102 is received, the load balancer parses the URL to browser as data transfer 104. 

determine which resource is being requested. Based on the The load balancer then enters a pass-through state. Any 

resource requested from parsing the URL, the load balancer 60 further packets from the browser such as ACK packets for 

determines which servers are best suited to serve the request the received packets are passed through to the assigned 

The load balancer then performs load balancing among the server. The assigned server normally closes the connection 

servers that can serve the request, step 125. immediately after the data has been sent to the browser. 

The load balancer then transfers the connection and the When the browser uses a "keep alive" mode, the server 

current TCP state to the assigned server, using TCP state 65 may not immediately close the connection. Additional 

migration 120. TCP state migration is not simply forwarding requests may be sent to the assigned server. FIG. 11B shows 

packets through as they are received. Instead the packets the browser sending a second URL request, PUSH(l), to the 
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load balancer, which is passed through to the assigned server to TCP/IP layer 92, perhaps after comparing checksums for 

as PUSH(1)\ The second request is then handled by the error detection. TCP/IP layer 92 is also modified to route 

server by sending the requested files to the browser, step SYN, ACK, RST, and FIN packets back to scheduler appli- 

126. Further requests are handled in a similar manner. cation layer 80 so the status of the connection can be 

Should one of these subsequent requests be for a file not 5 monitored, as explained later. Normal data transfers to 

located on the assigned server's local disk, a local network browsers use PUSH packets which are not routed to sched- 

can be used by the assigned server to access the requested uler application layer 80. 

files from other servers on the local network. TCP/IP layer 92 recognizes the virtual IP address as a 

Finally, the server closes the connection 130 by sending local IP address, so the packets are transferred up to appli- 

a finish (FIN) packet which is intercepted and sent to the 10 cation layer 90 after passing through the IP and TCP layers, 

load balancer, which passes it through to the client. The Application layer 90 runs the httpd server software which is 

client replies with an acknowledgment, ACK, and a FIN the standard web-site server software. Thus modified appli- 

packet indicating that the browser is closing its end of the cation server software is not needed, 
connection. The load balancer removes the session entry and FIG. 13 is a diagram of network layers transmitting data 

the server assignment, and the assigned server closes the is packets from the server to the client browser. The server 

connection. application in application layer 90 reads the files requested 

Protocol and IP Address Changes — FIGS. 12, 13 by the URL packets received in FIG. 12, and sends this data 

FIG. 12 is a diagram of network layers showing a packet to TCP/IP layers 92 addressed to the IP address of the client, 

sent from the client to the server which is intercepted by the which was the source IP address of the incoming packets and 

load balancer. A browser running on the application layer 75 20 was not changed. These packets use the TCP protocol and 

of the client sends packets to a web site to establish a the client's IP address C, and are designated "TCP(Q". 

connection and to send the URL requesting a file. The TCP/IP layer 92 uses the virtual IP address as the source 

browser application sends the data to be sent, the URL address of these outgoing packets rather than the real IP 

addressed to the virtual IP address V of the web site, to address of the server. Since the incoming packets have the 

TCP/IP stack 72. TCP/IP stack 72 is a stack of the standard 25 virtual IP address V as the destination IP address, the 

TCP and IP layers, which are roughly equivalent to the standard server software simply uses the destination IP 

session, transport, and network layers of the ISO network address of incoming packets as the source IP address of 

definition. TCP/IP stack packetizes the URL and adds head- outgoing packets. link layer 94 receives these packets and 

ers and sends packets to link layer 74, which is the driver for sends them out over the local media 76 and eventually 

the network-interface controller (NIC), link layer 74 sends 30 through Internet 66 to the client's link layer 74. These 

the packets out to Internet 66, perhaps through several packets are sent up through the client's TCP/IP stack 72 and 

routers, hubs, or switches (not shown). to the browser in application layer 75 where they are 

The TCP/IP packets sent from the client have the virtual displayed to the user. 
IP address V of the entire web site as their destination. These Since the client received the virtual IP address as the 

packets are shown as "TCP(V)" on the diagram. 35 source IP address in these packets, the client is unaware that 

The TCP(V) packets are received by the web site and the server is a different machine than the load-balancer, or 

routed to the node having the load balancer since the packets other servers at the web site, 

have the virtual IP address. These packets are received by Modified IP layer — FIG. 14 

the low-level link layer 84 and sent up to modified TCP/IP FIG. 14 is a diagram of modifications to the IP layer for 

stack 82. Modified TCP/IP stack 82 contains the standard 40 the load balancer's node. 

TCP and IP modules with some modifications explained Unmodified link layer 84 passes packets received up to 

later. One modification is that incoming packets from the TCP/IP stack 82, and specifically to IP input module 200 of 

Internet have their protocol changed from TCP to a propri- the IP layer. IP input module 200 determines if the packet is 

etary "IXP" protocol. Since this IXP protocol is unknown to destined for the local node or must be routed or forwarded 

the standard TCP and IP layers, it is sent directly up to 45 to another node. Forwarded packets are transferred to IP 

application layer 80 containing the load balancer. forward module 202, which prepares the packet for forward- 

The load balancer in application layer 80 receives the ing. Routing tables are consulted by forwarding module 202 

modified IXP(V) packets with the IXP protocol and the to determine where to sent the packet next. Forwarded 

virtual IP address V, and records the packets until the URL packets are then sent to IP output module 206, which sends 

is received when it performs load balancing. The packets are 50 them down to link layer 84. 

then played back to the assigned server to perform TCP state Local packets are assembled together for IP datagrams, 

migration as discussed in FIG. 11A. The packets played back which are passed up to TCP module 218 when the packet's 

are sent from application layer 80 as IXP(S) packets, having type or protocol is TCP. TCP module 218 transmits the 

the IXP protocol and the assigned server's IP address S. datagram to applications 212 in application layer 80 using 

Modified TCP/IP stack 82 uses the server's IP address S to 55 TCP socket 216. 

determine the NIC address and the routing information from Local packets that are not of a known protocol such as 

network routing tables. Once the routing has been deter- TCP or UDP (User Datagram Protocol) have an unrecog- 

mined and the NIC address of the assigned server is found, nized protocol. These datagrams are sent to raw socket 214, 

modified TCP/IP stack 82 changes the protocol back to TCP bypassing TCP module 218. Any applications in application 

and the IP address back to the virtual IP address V. The 60 layer 80 can listen to raw socket 214 and use the datagram, 

packets sent from modified TCP/IP stack 82 to link layer 84 since raw sockets are a standard TCP/IP feature. Load 

are TCP(V) packets. link layer 84 sends these packets out balancer 70 is an application which listens to raw socket 214 

over media 76 with the NIC address that corresponds to the for datagrams using the "IXP' protocol. Since the IXP 

assigned server, but with the virtual IP address as its desti- protocol is not a defined protocol, no other applications 

nation. 65 should be looking for IXP datagrams. Thus using the IXP 

The NIC address routes the TCP(V) packets to the protocol allows use of raw socket 214 to bypass the TCP 

assigned server's link layer 94, which passes the packets up layer and send the datagrams directly to load balancer 70. 
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These datagrams are the connection packets and the URL Ordinary TCP packets which are received from IP forward 

originally from the client's browser. module 202 or from an application and passed down through 

Each server is modified to accept packets using the virtual the TCP layer have their destination IP address read, step 

IP address by aliasing a second IP address, thus using two IP 330, and their route determined by accessing routing tables, 

addresses. For example, in UNIX, the command: 5 step 332. The NIC address of the destination or the next hop 

% ifconfig deO 230.101.17200 alias netmask OxffffffrT to the destination is determined and prefixed to the packet, 

specifies that a second IP address, the virtual IP address Ordinary TCP packets fail step 334 and are sent to the link 

230.101.17.200 is also an IP address for the node. Other layer, step 344. The link layer then transmits them over the 

operating systems also support IP address aliasing. media, unless the NIC address is the local node. These 

Modified IP Input Module— FIG. 15 10 local-destination packets are sent back up through the TCP/ 

FIG. 15 is a flowchart for a modified IP layer input IP stack, perhaps to a different application, 

module. The server with the load balancer uses modified IP Packets which have come from the load balancer appli- 

input module 200. An asterisk is used to indicate that the cation have the unrecognized IXP protocol rather than TCP, 

module is modified from the generic ip_jnputQ module. and are diverted by step 334 for special processing. When 

Steps 308, 310, 312, and 314 are added steps which are not 15 the load balancer assigns the server at the local node, step 

in the generic IP module. 336 detects that the destination is the local node, and the 

All packets received from the media by the lower link packets are passed to step 340, with the IXP protocol intact, 

layer are passed up to the IP layer which calls IP input These IXP packets are passed back up to modified IP input 

module 200. Step 302 tests to determine if the packet is for module 200 and are detected as being for the local server as 

the local node by reading the destination IP address. 20 discussed in the next section. 

When step 302 determines that the destination IP address Packets assigned to another node's server must be sent 

is not a local IP address, then the packet is being routed over the media to the assigned server. Since the assigned 

through the local node and the IP layer acts as a software servers all use standard TCP/IP stacks, the IXP protocol 

router. The packet is passed to IP forward module 202 (step must be changed back to TCP, step 338. The destination IP 

304) which prepares the packet for forwarding. The packet 25 address is set to the destination server's real IP address by 

is then sent to IP output module 206 before being the load balancer so that the destination's NIC address is 

re-transmitted out the link layer to the destination or the next generated by step 332 and prefixed to the packet. Since the 

hop. NIC address has already been determined, the IP address can 

Step 302 determines that the packet is for the local node safely be changed back to the virtual IP address of the entire 

when the IP address is the virtual IP address or the real IP 30 web site, step 340. The destination servers are all configured 

address for the server. The packet is stripped of its header to accept packets for the virtual IP address as well as for their 

information and possibly assembled with other packets to real IP address. Thus the real NIC address routes these 

form the IP datagram, step 306. packets to their destinations, which accepts them. 

The assembled IP datagram from step 306 is normally Modified Network Software for Wide-Area Network Sup- 
sent up to the TCP layer (steps 316, 318) for the generic IP 35 port 

module. The invention performs additional steps before step When the assigned server resides on the far side of a 
306 by modifying the generic IP input module to form router or across a wide-area network (WAN), the previous 
modified IP input module 200. Modified IP input module embodiment does not allow packets to get to their final 
200 checks the protocol to determine if it is the IXP protocol. destination. The load balancer normally transmits packets 
Since incoming packets from the Internet always use the 40 with the physical NIC address of the assigned server, but 
TCP protocol, incoming packets fail step 308 and are then with the virtual IP address. When the next hop is not the 
tested by step 310 to determine if they are TCP packets with assigned server, such as when the load balancer and the 
the virtual IP address and are world-wide-web packets. Thus assigned server are separated by a router, the router would 
step 310 looks for incoming packets. These incoming pack- route the packet back to the load balancer since the packets 
ets have their protocols changed from TCP to IXP, step 314. 45 have the load balancer's virtual IP address. 
The IXP protocol is not a recognized protocol, so step 316 FIG. 17 highlights that the real IP address of the assigned 
causes these incoming packets to be sent to the raw socket, server is used when multiple hops are required. Multiple- 
step 320, so that the load balancer application can read these hop data transmissions to the assigned server are supported 
packets. Thus changing the protocol to the unrecognized by further modifying the network software. The destination 
IXP protocol forces the incoming packets to be sent directly so IP address of the packets from the bad balancer to the 
to the load balancer. This allows all incoming packets from assigned server are further modified to have the assigned 
the Internet to be routed through the load balancer server's real IP address S rather than the virtual IP address 

Other TCP packets which are not world-wide web packets V. Thus intermediate routers can use the real IP address S of 

fail step 310 and are not modified. These ordinary TCP the assigned server to route the packet to the assigned server, 

packets are a known protocol, step 316, and are sent to the 55 When packets require multiple hops to reach the assigned 

TCP layer, step 318. server, the physical address of the next hop, intermediate 

Step 308, which first checks for the IXP protocol, is used router 97, is determined as before, but the real IP address of 

when the local node contains both the load balancer and the the assigned server is retained in the destination IP address 

assigned server. This step can be deleted if the local node is field of the IP header. To communicate the virtual address to 

used exclusively for the load balancer and cannot be an 60 the assigned server, the virtual IP address is appended to the 

assigned web server. end of the data in the packet and is sent to the assigned server 

Modified IP Output Module — FIG. 16 using the IXP protocol. Packets with the IXP protocol are 

FIG. 16 is a flowchart of an IP layer output module which intercepted and recovered by the assigned server, 

is modified for transmitting packets from the load balancer. A comparison of FIG. 17 to FIG. 12 shows that transmis- 

Modified IP output module 206 is a standard IP output 65 sion from load balancer's modified TCP/IP stack 82 to the 

module except that steps 334, 336, 338, and 340 have been assigned server's TCP/IP layer 92 uses IXP(S) packets 

added. rather than TCP(V) packets. Packets transmitted from the 
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load balancer to the assigned server are transmitted from 
link layer 84 of the load balancer over physical media 76 to 
intermediate router 97, then over physical media 76' to link 
layer 94 of the assigned server. Thus the local network at the 
server farm can use intermediate routers between the load 5 
balancer and the assigned servers. The assigned servers can 
also be located remotely from the load balancer, such as over 
a WAN using this technique. 
Added Steps When Assigned Server is Local 

Incoming packets which are assigned to the load balancer 10 
node's server are passed up and down the local TCP/IP stack 
twice. These packets are first sent from the low-level link 
layer through the modified IP layer to the load balancer in 
the application layer, and then back down through the IP 
layer to the link layer. Step 336 of FIG. 16 detects that the 15 
local server is the destination and bypasses steps 338, 340 so 
that the protocol is left as IXP. 

The link layer recognizes that the NIC address is the local 
NIC address and does not transmit the packets. Instead the 
packets are sent back up to the IP layer. Step 308 of FIG. 15 20 
detects these packets and changes the protocol back to TCP 
(step 312) and then passes the TCP packets to the HTTPD 
server application through the generic TCP layer. This 
sequence only occurs for a packet that has been intercepted 
to the load balancer and assigned to the server on the local 25 
node. 

IP Layer Modified for Servers 

The IP layers of the servers are modified in a similar way 
as the IP layer of the load balancer. All incoming packets are 
not affected, only special outgoing packets. These special 30 
packets are for establishing or closing a connection. These 
packets need to be intercepted and received by the load 
balancer during TCP state migration and when the session is 
closed. The load balancer keeps track of the session with an 
entry in a session table, and this entry is removed when the 35 
session ends. 

When the server closes a connection, it sends out a packet 
with the FIN flag set in the TCP header. When a transmission 
error occurs, a reset packet may be sent with the RST flag 
set. Likewise, when the server responds to the load balancer 40 
with the SYN/ACK packet during TCP state migration, the 
SYN and ACK flags are set. Normal data transfers to the 
browser are PUSH packets and do not have the SYN, ACK, 
RST, or FIN flags set. 

Hie server's IP output module is modified to detect these 45 
special packets by looking at the SYN, ACK, RST, and FIN 
flags. If none of these TCP flags are set, normal IP process- 
ing occurs. However, if any of these special TCP flags are 
set, then modified IP processing occurs. The protocol is 
changed from TCP to a variation of IXP called IXCP, and the 50 
destination address (the browser's IP address) is stored and 
replaced with the load balancer's IP address. The IP layer 
looks up the load balancer's IP address in its routing tables 
and generates the NIC address for the load balancer's node. 
The browser's IP address which was stored then replaces the 55 
load balancer's IP address, and the packet, with the IXCP 
protocol, is sent to the load balancer. 

The load balancer's IP layer processes these IXCP packets 
as shown in FIG. 15, the tests of steps 308, 310 fail, so that 
these IXCP packets are sent up to the load balancer through 60 
the raw socket. The load balancer can then alter its session 
table. FIN and RST packets are changed to TCP and the 
destination IP address changed to the browser to send out. 
SYN/ACK are not re-transmitted to the client. 
Load Balancer— FIG. 18 65 

FIG. 18 is a flowchart of the operation of the load 
balancer. The load balancer is written for the application or 
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user layer so that it can be easily modified. Writing the load 
balancer for a lower layer such as the IP layer puts the 
complex load balancing functions in the kernel, and any 
code modification could require that the system be shut 
down and rebooted. Having the load balancer in the appli- 
cation layer simplifies node management and allows code 
modifications to occur with minimal interruption. For per- 
formance reasons the inventors anticipate putting the load 
balancer into the kernel after being sufficiently debugged. 

Incoming web packets from the Internet are passed up to 
load balancer 70 from modified IP input module 200 (FIG. 
15). The load balancer is sometimes called a scheduler since 
it assigns or schedules sessions from browser clients to a 
server. 

The connection is first established with the browser client, 
step 350, by exchanging packets as was explained in more 
detail in FIG. 11A. These packets to the browser for estab- 
lishing the connection are TCP packets with the browser 
client's IP address; they are not affected by the modifications 
to the IP output module since the TCP protocol is designated 
for these connection packets. The incoming and outgoing 
connection packets are saved and later played back to the 
assigned server for TCP state migration, as was also shown 
in FIG. 11A. 

Once the connection is established with the client, load 
balancer 70 waits for the URL and then parses the URL to 
determine the requested resource. For more complex URL's, 
such as those containing coordinates of an icon, load bal- 
ancer 70 needs to decode the URL to determine what 
resource is being requested. This decoding is normally done 
by the HTTPD server software. The resource location is thus 
determined, step 352, and resource-based load balancing can 
now be performed, step 354. Load balancing occurs among 
those servers which contain the requested files or resource. 

The virtual IP address of the destination for each packet 
is changed to the real IP address of the assigned server, step 
356, before each packet played back to the assigned server 
is sent to modified IP output module 206. The real IP address 
is used to determine the real NIC address of the assigned 
server before the IP address is changed back to the virtual IP 
address by modified IP output module 206. AsendtoO call, 
step 358, is used to send the packets to modified IP output 
module 206. 

Fault-Tolerant Web Site— FIG. 19 

FIG. 19 is a diagram of a fault-tolerant web site with a 
back-up load balancer and dual Internet connections. 
Browser 10 sends requests through Internet 66 with a virtual 
IP address for the whole web site. Incoming packets with the 
virtual IP address are routed to load balancer 70 over local 
LAN 144. Local LAN 144 may contain routers, switches, 
and hubs when servers are located on separate network 
nodes. Local LAN 144 connects to Internet 66 through 
Internet connection 142 which directly connects to Internet 
connection router 140, and through Internet connection 148, 
which is connected to Internet connection router 146. 

Two separate connections 142, 148 are provided to Inter- 
net 66 to provide a backup when one connection fails, either 
due to line problems or failure of a connection router or 
other hardware or software. Having two connections 
increases the bandwidth which is needed for larger web 
sites. Each connection 142, 148 may be a slower Tl tele- 
communications connection, which supports 1.5 mega bits 
per second (Mbps), or a more powerful T3 connection which 
supports 45.5 Mbps. Other Internet connections may be used 
and mixed freely. Additional Internet connections may be 
added as demand increases simply by adding another con- 
nection router to connect the new connection to local LAN 
144. 
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Since the load balancer is no longer in the router, several level software program in the application layer which is 

connection routers may be used and added or subtracted as more easily modified than lower-level kernel modules, 

needed without affecting load balancing. Standard routers The standard TCP/IP layers are used except for modifi- 

and new technologies can be used. The connection router is cations to the IP layer. This is an advantage since standard 
not limited to having a certain operating system used by the 5 software is used as much as possible. The application layer 

load balancer. Should a connection router 140, 146 fail, a runs the standard httpd server software. Modified server 

backup connection router can continue to connect the web software is not needed. Having the load balancer in the 

servers to Internet 66. There is no single point of failure. application layer simplifies node management and allows 

A backup load balancer 70' is also provided to take over code tweaks to occur with minimal interruption, 

operation should primary load balancer 70 fail These load 10 ALTERNATE EMBODIMENTS 

balancers are located on separate servers to lessen the Sevend otfaer embodiments are contemplated by the 

chance that both fail at the same time. Backup load balancer mventors . For ex le various local networks ^ ^ 

70 closely monitors primary load balancer 70 to detect a tQ0SC which ^ or othcr mc addrcsscs 

failure. rather than MAC address as the local or physical network 

Tht content of the web site is not mirrored to each is address Switches> ^ ^ hardware £ software 4>ased 
server's disk. However, to prevent the failure of one server routefS caQ ^ inserted fof { netwo ^ 

from making some content unavailable, at least one backup ^ , 4 r w . . .. 4 

j c . n . . iL , . ^ , r Each server can process requests from multiple clients, 

copy .s made of each file m the web s,tc. For example, ^ m ^ ltitaski 4 C ratin such ^ 

file .html 26 exists on disk 62 attached to server 52, and a TT kttv j «r j xr-r , 

, , - ni . . „ . , 4 , . , n ' UNIX and Windows NT are used. While a connection 

backup copy of file. html 26 is located on disk 60 of server 20 4 . , t . . T . t . , , ~ . 

*i r j l 1 *n • * • * i_i *u j * *_ through the public Internet has been described, the connec- 

51. Load balancer 70 maintains a table or other data struc- A . , ■, T . J4 . • . . . 

ff « A . t fC1 . t , , . u . . . , tion could also be made through private networks such as 

ture of all the locations of files in the web site which is used 4 r* * » * * 

* * j i_ 1 * 0 . j 4 corporate networks known as Intranets. Intranets are just a 

for load balancing. Software utilities may be used to repk- u * * *u 1 1 * * -m. *u . J ,j L 

,ci j 1 i 11 * u * subset of the larger Internet. Thus the web site could be 

cate new files or delete all copies of old files. Access . . r n « j *i_ • -li * r 

4 4 . r . j- * * in u 1 behind a corporate firewall and not be visible to the users of 
statistics of each file or directory of files can be kept to 25 me j atemet 

determine which files are more frequently used and should _ * „ , , 

be replicated to more servers, and which files are infre- . ™e web site may be a web-hoster* containing many web 

quently used and only two copies should be kept. Sltes for dlfferc f ^?i mes ra f her a ^website. To 

Content A is located on disk 58 attached to server 56, «W?t a variety of different sites the load balancer may be 

which is used for both load balancing and as a server. 30 * » acce £ mukl P le v^al IP adoYesses for support 

Content A is also located on disk 60 attached to server 51. of "*ul* ; homed servers. Heterogeneous clusters could 

Content B is located on disk 60 and disk 62, while content J?*}™ dlfferent tv P es of ^J^VT^i 

C is located on disk 58 and disk 63 accessed by server 55, ^"l^^ SystcmS (0S s) ™ ch ™ UNIX ' Wmdows/ 

which also contains backup load balancer 70'. ^ SOLARIS, etc. 

A failure of one disk or one server does not crash the 35 ^ web farm has been described as havmg a 'local' 

entire web site, or even make unavailable some of its network, but this local network could be local only in the 

content, as all content is backed up on at least one other s* 0 ** that u * not ^ Intemet backbone. Servers in the web 

server. When a failure occurs, system maintenance software ma y 1x5 geographically remote, where some of the 

makes another backup copy of the lost files so that a second t*™™ m located in one city while other servers are located 

failure does not make some files unavailable. 40 m other dlies - lMd balancing may be performed not just 

Even a failure by load balancer 70 does not bring down based on content, but also geographically to minimize traffic 

the web site as backup load balancer 70* is ready to take over on me network backbone. The parts of the web site m a city 

load balancing. A failure by the Internet connection router mav be connected locally through one or more LAN's, while 

also does not lock out the entire web site as a backup bein S connected to other cities using a WAN. The IXP 

connection and router can be used. 45 protocol can be used for all packets sent from the load 

balancer to the assigned server, even when multiple hops are 

ADVANTAGES OF THE INVENTION not required. 

Request-level load-balancer granularity is provided by the The invention has been described as a browser accessing 
web site since a browser sees a failure only when the a file on the server, but the file may actually be a resource 
assigned server fails while processing the URL request 50 which is not necessarily a traditional file. The file may be a 
Since the entire web site uses a virtual IP address, client program, applet, or script which is executed, or an interface 
caching of the IP address causes no problems, as all new into an SQL database or to fast or memory-intensive corn- 
requests are routed to the load balancer, or the backup load putational resource. Web servers support application- 
balancer when the primary load balancer fails. Programming Interfaces (API's) which enable servers to be 

The invention provides a highly fault-tolerant web site. 55 compiled with unique capabilities for alternate types of 

Such a web site greatly reduces the probability that a user server resources. 

gets the "SERVER NOT RESPONDING" error message Since these resources are expensive to implement on all 

when a server fails at the web site. Request-level granularity servers, it is desired to allocate just one or a few servers to 

results in fewer browser users being caught when a server handling requests for these resources. Hie invention waits to 

crashes; only those users having a current request being 60 perform load-balancing and assign a server until after the 

served by that server experience an error. Other requests in URL has been parsed. Thus the invention is ideal for 

a session at that web site, and future sessions are not assigning specialty servers having these resources. The web 

affected. site can be segregated by resource and still be load-balanced. 

Servers and routers and even Internet connections can be The invention may also be applied to other Internet traffic 

added or removed without bringing down the web site. Thus 65 as well. The invention could be applied to a File-Transfer- 

a highly maintainable and expandable web site is continu- Protocol (FTP) server, a rlogin server, or a telnet server, 

ously available to users. The load balancer itself is a user- either as a stand-alone site or as a part of a larger web site. 
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While delayed resource binding is preferred, other 
embodiments are contemplated. HTTP redirection from the 
scheduler to the assigned server may be used after the URL 
has been parsed. The load balancer sends the client the 
address of the assigned server and instructs the client to 
re-issue the URL request using the assigned server's 
address. Thus redirection is delayed until the URL is parsed 
and the requested content is determined. 

The URL passed to web server can also be modified to 
reference relocated web pages without changing the links. 

The foregoing description of the embodiments of the 
invention has been presented for the purposes of illustration 
and description. It is not intended to be exhaustive or to limit 
the invention to the precise form disclosed. Many modifi- 
cations and variations are possible in light of the above 
teaching. It is intended that the scope of the invention be 
limited not by this detailed description, but rather by the 
claims appended hereto. 

We claim: 

1. A web site for sending resources to a browser on a client 
connected to a computer network, the web site comprising: 

a network connection point for receiving incoming data 
packets from the computer network and for transmit- 
ting outgoing data packets to the computer network; 

local network, coupled to the network connection point, 
for transferring data packets; 

a plurality of network nodes containing web servers with 
resources, the plurality of network nodes connected to 
the local network, the plurality of network nodes 30 
including means for transmitting the resources as out- 
going data packets to the client, the plurality of network 
nodes including means for sending the outgoing data 
packets over the local network to the network connec- 
tion point; 

wherein the plurality of network nodes containing web 
servers together contain all resources at the web site, 
but each network node in the plurality of network nodes 
contains only a portion of all the resources at the web 
site; 

a balancer network node containing a load balancer, 
receiving the incoming data packets transmitted over 
the local network from the network connection point, 
the load balancer for determining an assigned server in 
the plurality of network nodes for responding to a 
request from the client in an incoming data packet, the 
load balancer including means for transferring a con- 
nection to the client to the assigned server; 
wherein the balancer network node containing the load 
balancer is connected to the network connection point 
by the local network which is also connected to the 
plurality of network nodes, 
wherein network nodes are segregated to contain different 
resources, and wherein all resources at the web site are 
not mirrored to all network nodes at the web site, 
wherein the load balancer further comprises: 
content means for storing an indication of which net- 
work nodes in the plurality of network nodes contain 
each resource; 
URL means, receiving incoming data packets from the 
client containing a request for a resource, for deter- 
mining a requested resource from the incoming data 
packets; 

compare means, coupled to the content means and 
coupled to the URL means, for comparing the 
requested resource to the indication of which net- 
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work nodes in the plurality of network nodes contain 
each resource, and for outputting a list of network 
nodes containing the requested resource; 
balancing means, receiving the list of network nodes 
containing the requested resource, for choosing as an 
assigned node one of the network nodes in the list of 
network nodes, 
whereby the incoming data packets are routed to the 
balancer network node but outgoing data packets 
bypass the balancer network node and whereby the load 
balancer chooses an assigned node based on the 
resources contained by each network node, the load 
balancer performing resource-based load balancing. 

2. The web site of claim 1 wherein the balancer network 
node is in the plurality of network nodes containing web 
servers. 

3. The web site of claim 1 wherein the web site is 
addressable by one network address for all web servers in 
the plurality of network nodes containing web servers. 

4. The web site of claim 1 further comprising: 

delay means, in the load balancer, for delaying assignment 
of the assigned node until an incoming data packet 
containing the request for the resource is received, 

whereby load balancing is delayed. 

5. The web site of claim 1 further comprising: 
redirect means, in the load balancer, for directing the 

client to issue a new URL request directly to the 
assigned node using an address of the assigned node 
provided by the load balancer to the client; 
whereby the client is redirected to the assigned server by 
the load balancer. 

6. A computer-implemented method of servicing requests 
for resources from a client by nodes containing different 
resources, the computer-implemented method comprising 
the steps of: 

making a connection and setting up a session between the 
client and a load balancer at a web site for servicing 
requests from clients; 

waiting for a URL request from the client once the load 
balancer has made the connection with the client; 

receiving the URL request from the client and decoding 
the URL request to determine a requested resource; 

comparing an identifier for the requested resource to 
identifiers for resources located on a plurality of nodes 
and determining a first subset of the plurality of nodes 
which contain the requested resource and a second 
subset of the plurality of nodes which do not contain the 
requested resource; 

assigning the URL request to an assigned node in the first 
subset of the nodes which contain the requested 
resource, by determining the assigned node to be a 
server in the first subset of the nodes which is least busy 
processing requests, wherein the assigned node is not in 
the second subset; 

transferring the connection and the session setup to the 
assigned node containing the requested resource by 
storing packets received from the client when estab- 
lishing the connection and by transmitting the pack- 
ets to the assigned node after the URL request is 
received; 

reading the requested resource on the assigned node and 
transmitting the requested resource to the client, 

whereby the assigned node is selected based on a location 
of the requested resource determined from the URL 
request and load balancing is performed among nodes 



06/05/2003, EAST Version: 1.03.0002 



5,774, 

23 

having the requested resource and the connection is 
transferred from the load balancer to the assigned node 
by re-transmitting the packets to the assigned node. 

7. The computer-implemented method of claim 6 wherein 
the packets received from the client are TCP/IP packets 5 
having a destination IP address being a virtual IP address of 
the load balancer, and wherein the step of transmitting the 
packets to the assigned node comprises: 

changing the virtual IP address of the load balancer in the 
packets to a real IP address of the assigned node and 10 
passing the packets to a modified IP layer; 

determining from the real IP address a physical route from 
the load balancer to the assigned node over a network 
and generating a physical network address for the 
assigned node and attaching the physical network 15 
address to the packets; 

changing the real IP address in the packets back to the 
virtual IP address before transmission of the packets 
with the physical network address, 

whereby the physical network address is generated from 
the real IP address of the assigned node, but the packets 
transmitted to the assigned node contain the virtual IP 
address of the load balancer. 

8. Hie computer-implemented method of claim 6 wherein ^ 
the packets received from the client are TCP/IP packets 
having a destination IP address being a virtual IP address of 
the load balancer, and wherein the step of transmitting the 
packets to the assigned node comprises: 

changing the virtual IP address of the load balancer in the 30 
packets to a real IP address of the assigned node and 
passing the packets to a modified IP layer; 

determining from the real IP address a physical route from 
the load balancer to an intermediate router in a path to 
the assigned node over a network and generating a 35 
physical network address of the intermediate router and 
attaching the physical network address of the interme- 
diate router to the packets; and 

transmitting packets having the real IP address of the 
assigned node as the destination IP address and the 40 
virtual IP address of the load balancer appended to data 
in the packet; 

recovering the virtual IP address of the load balancer from 
the data in the packet when the packet is received by the 
assigned node, 45 

whereby the physical network address of the intermediate 
router is generated from the real IP address of the 
assigned node, the load balancer and the assigned node 
being separated by the intermediate router. 

9. The computer-implemented method of claim 7 wherein 50 
the load balancer is a program in an application layer above 

a TCP layer which is above the modified IP layer which is 
above a link layer, wherein the step of receiving the URL 
request from the client comprises: 

receiving at least one TCP/IP packet from the client and 
assembling an IP datagram from the at least one TCP/IP 
packet in the modified IP layer; 
changing a protocol for the IP datagram from TCP to an 

unrecognized protocol; 60 
bypassing the TCP layer and transmitting the IP datagram 
to the load balancer in the application layer through a 
raw IP socket, 
whereby the TCP layer is bypassed for incoming TCP/IP 
packets of the URL request. 65 

10. The computer- implemented method of claim 9 
wherein the step of transferring the connection and the 
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session setup to the assigned node containing the requested 

resource further comprises: 
passing the packets with the virtual IP address up through 
a modified IP layer and a standard TCP layer to a 
standard server application in an application layer on 
the assigned node, the assigned node being configured 
to accept packets with either the real IP address of the 
assigned node or the virtual IP address of the load 
balancer, 

whereby the assigned node uses the modified IP layer and 
the standard server application. 

11. The computer-implemented method of claim 10 
wherein the step of transmitting the requested resource to the 
client from the assigned node comprises 

transmitting the requested resource in TCP/IP outgoing 
packets which contain the virtual IP address of the load 
balancer as a source IP address but an IP address for the 
client as the destination IP address, wherein the TCP/IP 
outgoing packets bypass a node with the load balancer, 

whereby incoming packets are routed to the load balancer 
but the outgoing packets bypass the node with the load 
balancer. 

12. The computer-implemented method of claim U fur- 
ther comprising the steps of: 

creating a session entry for the client in the load balancer 
when the URL request from the client is received by the 
load balancer; 

updating the session entry for the client to indicate the 
assigned node when the load balancer assigns the URL 
request to the assigned node, 

whereby the load balancer tracks sessions between clients 
and assigned nodes. 

13. The computer-implemented method of claim 12 fur- 
ther comprising the steps of: 

reading a FIN flag in the TCP/IP outgoing packets and 

determining that the TCP/IP outgoing packet is a FIN 

packet when the FIN flag is set; 
changing the IP address of the client to the virtual IP 

address of the load balancer as the destination IP 

address for the FIN packet; 
transmitting the FIN packet to the load balancer and 

closing the session entry for the client in the load 

balancer in response to the FIN packet; 
re-transmitting from the load balancer the FIN packet to 

the client, 

whereby FIN packets are intercepted by the load balancer. 

14. A fault-tolerant server farm for serving resources to 
browser clients remotely located on a network, the resources 
containing links to other resources not located at the server 
farm but located on distant computers on the world-wide 
web, each link being a universal-resource locator (URL), the 
URL indicating a host name and a requested resource, the 
host name indicating a server farm on the network contain- 
ing the requested resource, the fault-tolerant server farm 
comprising: 

a network connection for transferring packets from the 
network to a local network; 

a plurality of nodes, each node being a computer con- 
taining a disk and a connection to the local network; 

a plurality of frequently-accessed resources stored on the 
disk for each node; 

a plurality of less-frequently-accessed resources, each of 
the less-frequently-accessed resources stored on disks 
for at least two nodes but not stored on the disk for each 
node; 
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a primary load balancer, residing on a primary node in 
the plurality of nodes, for receiving all incoming 
packets from the network connection, the primary 
load-balancer assigning URL requests from browser 
clients to nodes in the plurality of nodes, wherein the 
primary load balancer comprises: 
storage means for storing at least a portion of con- 
nection incoming packets for establishing a con- 
nection between a browser client and the server 
farm; 

reply means for generating acknowledgment packets 
to the browser client in response to the connection 
incoming packets; 

URL decoder means, receiving a URL packet once 
the connection with the browser client is made, for 
decoding the URL to determine a requested 
resource requested by the browser client; 

assignment means for selecting an assigned node in 
the plurality of nodes by not selecting nodes which 
have disks which do not contain the requested 20 
resource; 

transfer means for transferring the connection to the 
assigned node by constructing packets using the 
storage means which stored at least a portion of 
connection incoming packets; 25 

pass-through means for transferring incoming pack- 
ets from the browser client to the assigned node 
once the connection has been transferred to the 
assigned node, 
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a secondary load balancer, residing on a secondary node 
in the plurality of nodes, for receiving all incoming 
packets from the network connection when the primary 
load balancer fails, the secondary load-balancer assign- 
ing URL requests from browser clients to nodes in the 
plurality of nodes, 

whereby each node does not contain all resources at the 
server farm and the primary and secondary load bal- 
ancers reside on nodes connected to the local network. 

15. The fault-tolerant server farm of claim 14 further 
comprising: 

balancing means, coupled to the primary load balancer 
and to the secondary load balancer, for assigning con- 
nection incoming packets to either the primary load 
balancer or to the secondary load balancer, 

whereby load balancing is distributed between the pri- 
mary load balancer and the secondary load balancer. 

16. The fault-tolerant server farm of claim 15 wherein the 
network is the Internet, the fault-tolerant server farm further 
comprising: 

a secondary Internet connection for transferring packets 
from the Internet to a local network, 

whereby two Internet connections connect the local net- 
work to the Internet, 
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